Pattern matching based character string retrieval

ABSTRACT

Embodiments relate to generating a retrieval condition for retrieving a target character string from texts by pattern matching. An aspect includes dividing a first text into words. Another aspect includes generating a converted character string by performing at least one of appending at least one character in at least either one of previous and subsequent positions of the target character string. Another aspect includes replacing at least one character of the target character string. Another aspect includes generating the retrieval condition for retrieval candidates in the words of the first text, the retrieval condition comprising determining that a retrieval candidate matches the target character string and does not match the converted character string based on a ratio of a part of the retrieval candidate which matches the converted character string and corresponds to the target character string is less than or equal to a reference frequency.

BACKGROUND

The present disclosure relates generally to information processing, andmore specifically, to pattern matching based character string retrieval.

An information processing device may extract character strings from adatabase that stores character strings, and then exclude some of theextracted character strings. For example, an information processingdevice may extract some character strings such as, for example,“development cost,” “cost,” and “development” from a database ofcharacter strings in which text is organized in records. Thereafter, thedevice may delete “development cost”, which overlaps “development”, and“cost”, and creates a new database including character strings“development” and “cost.” However, a lot of required character stringsmay be incorrectly excluded since the information processing deviceexcludes character strings composed of some of a plurality of extractedcharacter strings, in other words, combined words composed of aplurality of character strings, and therefore character strings whichshould be extracted are not extracted, which may lead to retrieval onlybeing allowed under retrieval conditions with low accuracy of extractingcharacter strings.

SUMMARY

Embodiments relate to generating a retrieval condition for retrieving atarget character string from texts by pattern matching. An aspectincludes dividing a first text into words. Another aspect includesgenerating a converted character string by performing at least one ofappending at least one character in at least either one of previous andsubsequent positions of the target character string. Another aspectincludes replacing at least one character of the target characterstring. Another aspect includes generating the retrieval condition forretrieval candidates in the words of the first text, the retrievalcondition comprising determining that a retrieval candidate matches thetarget character string and does not match the converted characterstring based on a ratio of a part of the retrieval candidate whichmatches the converted character string and corresponds to the targetcharacter string is less than or equal to a reference frequency.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention. For a better understanding of the invention with theadvantages and the features, refer to the description and to thedrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments will now be described, by way of example only, withreference to the following drawings in which:

FIG. 1 illustrates an embodiment of an information processing device.

FIG. 2 illustrates an embodiment of flowchart of retrieval conditiongeneration processing and retrieval processing performed by aninformation processing device.

FIG. 3 illustrates an example of a Venn diagram describing thegeneration of a retrieval condition.

FIG. 4 illustrates an embodiment of a display image for selectingexclusion candidates.

FIG. 5 illustrates an embodiment of a targeted text corpus forretrieval.

FIG. 6 illustrates an embodiment of a text corpus including a third textused for the generation of a retrieval condition.

FIG. 7 illustrates an embodiment of a hardware configuration of acomputer.

DETAILED DESCRIPTION

Embodiments of pattern matching based character string retrieval aredisclosed herein, with exemplary embodiments being discussed below indetail. According to an embodiment, there is provided an informationprocessing device which generates a retrieval condition for retrieving atarget character string from texts by pattern matching, the devicecomprising: a language processing unit which divides a first text intowords by language processing; a character conversion unit whichgenerates a converted character string by performing at least one ofappending at least one character in at least either one of previous andsubsequent positions of the target character string and replacing atleast one character of the target character string; and a conditiongeneration unit which generates the retrieval condition of matching thetarget character string and not matching the converted character stringon condition that the ratio of a part which matches the convertedcharacter string and corresponds to the target character string amongthe words divided by the language processing is equal to or less than areference frequency in the first text. According an embodiment, there isprovided a method for the information processing device. Furthermore,according to an embodiment, there is provided a computer program productfor the information processing device.

FIG. 1 illustrates an embodiment of an information processing device 10.The information processing device 10 generates a retrieval condition 34for an accurate extraction of words of a target character string to beretrieved by language processing through character string retrieval. Anexample of the information processing device 10 is a computer such as apersonal computer or the like.

The information processing device 10 includes a control unit 12, adisplay unit 14, an input unit 16, and a storage unit 18. Instead,without including any one of the display unit 14, the input unit 16, andthe storage unit 18, the information processing device 10 may use adisplay unit 14, an input unit 16, or a storage unit 18 provided in anexternal device.

The control unit 12 is an arithmetic processing unit such as a centralprocessing unit (CPU). The control unit 12 includes a languageprocessing unit 22, a character conversion unit 24, a conditiongeneration unit 26, and a retrieval unit 28. For example, the controlunit 12 may be configured to function as the language processing unit22, the character conversion unit 24, the condition generation unit 26,and the retrieval unit 28 by reading a program for retrieval conditiongeneration processing and a program for retrieval processing from thestorage unit 18 or via a network. In addition, some or all of thelanguage processing unit 22, the character conversion unit 24, thecondition generation unit 26, and the retrieval unit 28 may be composedof hardware such as circuits or the like.

The language processing unit 22 divides a first text into words bylanguage processing. For example, the language processing unit 22 isconnected to the storage unit 18 and to the condition generation unit26. The language processing unit 22 acquires one or more first textsincluded in a text corpus 32 for learning stored in the storage unit 18and divides the first text concerned into words by language processing.The language processing unit 22 may divide the first text into words onthe basis of a morphological analysis using words and a grammarregistered in dictionary data 30. The language processing unit 22outputs the first text divided into words to the condition generationunit 26.

The character conversion unit 24 appends at least one character in atleast either one of the previous and subsequent positions of a targetcharacter string to generate a converted character string. For example,the character conversion unit 24 is connected to the storage unit 18 andto the condition generation unit 26. The character conversion unit 24generates a converted character string, in which at least one characteris appended in either one of the previous and subsequent positions ofthe target character string, on the basis of the words acquired from thedictionary data 30 in the storage unit 18. The character conversion unit24 outputs the target character string, the converted character string,and the like to the condition generation unit 26.

The condition generation unit 26 is connected to the language processingunit 22, the character conversion unit 24, and the storage unit 18. Thecondition generation unit 26 acquires the first text, which has beendivided into words, from the language processing unit 22. The conditiongeneration unit 26 acquires the target character string and theconverted character string from the character conversion unit 24. Thecondition generation unit 26 generates a retrieval condition 34 ofmatching the target character string and not matching the convertedcharacter string on condition that the ratio of a part which matches theconverted character string and corresponds to the target characterstring among the words divided by the language processing is equal to orless than a reference frequency in the first text. This enables thecondition generation unit 26 to generate the retrieval condition 34 forretrieving the target character string by character string retrieval andfor retrieving words including a converted character string to bedistinguished as a different word. The condition generation unit 26causes the storage unit 18 to store the generated retrieval condition34. The condition generation unit 26 outputs image information of adisplay image including an exclusion candidate, which is a candidate notto be matched with the converted character string, to the display unit14.

The retrieval unit 28 retrieves a text on the basis of the retrievalcondition 34 generated by the condition generation unit 26. For example,the retrieval unit 28 retrieves a text including a character stringmatching the retrieval condition 34 within the text corpus 32 andextracts the text.

The display unit 14 displays an image on the basis of the imageinformation obtained from the condition generation unit 26 of thecontrol unit 12. An example of the display unit 14 is an organicelectroluminescent (EL) display device or a liquid crystal displaydevice.

The input unit 16 accepts an input from a user and outputs it to thecontrol unit 12. An example of the input unit 16 is a keyboard, a mouse,a touch panel, or the like.

The storage unit 18 stores programs executed by the control unit 12 andrequired information such as parameters or the like in the execution ofthe programs. For example, the storage unit 18 stores the program forretrieval condition generation processing and the program for retrievalprocessing. The storage unit 18 stores the dictionary data 30 and thetext corpus 32 that is used to execute the program for retrievalcondition generation processing and the program for retrievalprocessing. The dictionary data 30 may be, for example, data in aJapanese dictionary or may be dictionary data of medical or othertechnical terminology. The text corpus 32 may be, for example, adatabase including general texts or may be a database including texts inmedical or other specific fields. An example of a text in the medicalfield is a sentence described in an insurance application form.

FIG. 2 is a flowchart of an embodiment of a retrieval conditiongeneration processing and retrieval processing performed by theinformation processing device 10. FIG. 3 is a Venn diagram thatillustrates the generation of the retrieval condition 34. FIG. 4 is adiagram illustrating a display image 40 for selecting exclusioncandidates. FIG. 5 is a diagram illustrating a targeted text corpus 32for retrieval. The control unit 12 performs the retrieval conditiongeneration processing and the retrieval processing by reading theprogram for retrieval condition generation processing and the programfor retrieval processing. In this embodiment, a target character stringto be retrieved is assumed to be “ga-n” (cancer, in Japanese).

As illustrated in FIG. 2, in the retrieval condition generationprocessing, first, the character conversion unit 24 acquires a targetcharacter string (block S10). For example, the character conversion unit24 acquires a target character string from the words registered in thedictionary data 30 stored in the storage unit 18. The characterconversion unit 24 may acquire a target character string through auser's input from a keyboard or the like.

The character conversion unit 24 generates a converted character stringby appending at least one character in at least either one of theprevious and subsequent positions of the target character string (blockS12). For example, the character conversion unit 24 retrieves a wordpartially including the target character string from the wordsregistered in the dictionary data 30 used for language processing andgenerates a converted character string by appending at least onecharacter located in at least either one of the previous and subsequentpositions of the target character string with respect to the retrievedword concerned. Furthermore, the character conversion unit 24 maygenerate a converted character string by appending at least onecharacter, which is common to words whose number is greater than orequal to a predetermined reference number of words Sta among a pluralityof words partially including the target character string in thedictionary data 30 used for language processing, to the target characterstring.

The following illustrates an embodiment of the generation of theconverted character string performed by the character conversion unit 24by using specific examples. It is assumed that the character conversionunit 24 extracts words “ga-n” (cancer), “yuu-hatsu-sei-ga-n” (inducedcancer), “yuu-hatsu-sei-ga-n” (induced cancer), “ga-n-yuu-hatsu-sei”(cancer-induced), “ga-n-yuu-hatsu-sei” (cancer-induced), “ga-n-ken-shin”(cancer medical examination), “ga-n-ken-shin” (cancer medicalexamination), and “ga-n-sai-bou” (cancer cell) as a result of retrievingwords including a target character string “ga-n” (cancer) from thedictionary data 30. The words may be extracted from entries (namely,headwords) of the dictionary data 30.

For example, the character conversion unit 24 counts the number ofcharacter strings “sei-ga-n” in which one character is added to thetarget character string “ga-n” (cancer) in the previous position thereofin the word “yuu-hatsu-sei-ga-n” (induced cancer). The dictionary data30 contains the words “yuu-hatsu-sei-ga-n” (induced cancer) and“yuu-hatsu-sei-ga-n” (induced cancer) and therefore the characterconversion unit 24 counts the number of character strings “sei-ga-n” astwo. Here, the reference number of words Sta is set to two. The numberof character strings “sei-ga-n” is greater than or equal to thereference number of words Sta, and therefore the character conversionunit 24 further counts the number of character strings “hatsu-sei-ga-n”in which one character is added in the previous position of thecharacter string “sei-ga-n” of the word “yuu-hatsu-sei-ga-n” (inducedcancer). The number of character strings “hatsu-sei-ga-n” is two in thesame manner and greater than or equal to the reference number of wordsSta. Therefore, the character conversion unit 24 further counts thenumber of character strings “yuu-hatsu-sei-ga-n” (induced cancer). Thenumber of character strings “yuu-hatsu-sei-ga-n” (induced cancer) isalso two and greater than or equal to the reference number of words Sta.Therefore, the character conversion unit 24 counts the number ofcharacter strings “yuu-hatsu-sei-ga-n” (induced cancer). A wordincluding “yuu-hatsu-sei-ga-n” (induced cancer) is only“yuu-hatsu-sei-ga-n” (induced cancer), and therefore the characterconversion unit 24 counts the number of the character strings concernedas one. The number of character strings is less than the referencenumber of words Sta and therefore the character conversion unit 24 doesnot consider the character string “yuu-hatsu-sei-ga-n” (induced cancer)as a converted character string.

On the other hand, since “yuu-hatsu-sei-ga-n” (induced cancer) satisfiesthe condition that “the number of character strings is greater than orequal to the reference number of words Sta, the character conversionunit 24 considers the character string “yuu-hatsu-sei-ga-n” (inducedcancer) as a converted character string. In other words, the characterconversion unit 24 appends three characters “yuu-hatsu-sei” (induced) inthe previous position of the target character string “ga-n” to generatea converted character string “yuu-hatsu-sei-ga-n” (induced cancer).Here, the character conversion unit 24 considers one or more charactersappended to the target character string to be a retrieval candidate. Theretrieval candidate in this specification is the longest characterstring “yuu-hatsu-sei” (induced) among the characters whose number isgreater than or equal to the reference number of words Sta appended tothe target character string “ga-n” (cancer). The character conversionunit 24 enables the discrimination between the previous position and thesubsequent position of the target character string where the retrievalcandidate is appended to the target character string.

Similarly, the character conversion unit 24 counts the number ofcharacter strings “ga-n-yuu” in which one character is added in thesubsequent position of the target character string “ga-n” (cancer) inthe word “ga-n-yuu-hatsu-sei” (cancer-induced) in the dictionary data30. The number of the character strings concerned is counted as two,which is greater than or equal to the reference number of words Sta,based on the word “ga-n-yuu-hatsu-sei” (cancer-induced) and the word“ga-n-yuu-hatsu-sei” (cancer-induced). Thereafter, the characterconversion unit 24 performs the same processing as the above to generatea character string “ga-n-yuu-hatsu-sei” (cancer-induced) as a convertedcharacter string and to consider the character string “yuu-hatsu-sei”(induced) as a retrieval candidate. Similarly, the character conversionunit 24 generates a character string “ga-n-ken-shin” (cancer medicalexamination) as a converted character string from the word“ga-n-ken-shin” (cancer medical examination) and the word“ga-n-ken-shin” (cancer medical examination) in the dictionary data 30and considers the character string “ga-n-ken-shin” (cancer medicalexamination) as a retrieval candidate.

On the other hand, a character string “ga-n-sai” in which one characteris added in the subsequent position of the target character string“ga-n” (cancer) in the word “ga-n-sai-bou” (cancer cell) in thedictionary data 30 does not overlap other words. Therefore, thecharacter conversion unit 24 counts the number of character stringsconcerned as one and determines that the number is less than thereference number of words Sta. Therefore, the character conversion unit24 does not generate the word “ga-n-sai” as a converted characterstring.

The character conversion unit 24 outputs the converted character string,the target character string, and the retrieval candidate to thecondition generation unit 26.

The language processing unit 22 divides the first text into words bylanguage processing (block S14). For example, the language processingunit 22 acquires one or more first texts from the text corpus 32 storedin the storage unit 18. The language processing unit 22 divides theacquired first texts into words on the basis of the words alreadyregistered in the dictionary data 30 stored in the storage unit 18. Thelanguage processing unit 22 outputs one or more first texts divided intowords to the condition generation unit 26.

The condition generation unit 26 generates a set, which is illustratedin FIG. 3, of converted character strings included in the first texts(block S16). For example, the condition generation unit 26 retrieves apart matching the converted character string included in the firsttexts, in other towards, a part coincident with the converted characterstring as a matter of the character string without regard for thedivision into words by the language processing unit 22 by characterstring retrieval and extracts the coincident parts. Therefore, thecondition generation unit 26 retrieves and extracts all parts matchingthe converted character string in the first texts.

The condition generation unit 26 generates a set of words coincidentwith the target character string among the words into which the firsttexts are divided by the language processing (block S18). Specifically,the condition generation unit 26 retrieves and extracts a partcoincident with the target character string in the first texts dividedinto words by the language processing unit 22 and then generates the setillustrated in FIG. 3.

The condition generation unit 26 determines whether the ratio of a partwhich matches the converted character string and corresponds to thetarget character string among the words divided by the languageprocessing in the first text satisfies the condition of the referencefrequency or less (block S20). For example, the condition generationunit 26 determines whether the ratio of a part which matches theconverted character string satisfies the condition of the referencefrequency or less, among the parts corresponding to the target characterstring in the words divided by the language processing in the firsttexts. The reference frequency is a numerical value between 0 and 1 suchas, for example, 0.5. Specifically, the condition generation unit 26determines whether the following expression (1) is satisfied, wherein This the reference frequency, Ra is the set of converted characterstrings, and Rx is the set of words:Th≥#(Ra∩Rx)/#Rx  (1)

(Ra∩Rx) indicated by hatching as illustrated in FIG. 3 is an overlappingarea between the set Ra and the set Rx. The symbol # indicates thenumber of character strings or words in the set.

If determining that the expression (1) is not satisfied, specifically,that the ratio of the part which matches the converted character stringand corresponds to the target character string among the words dividedby the language processing is greater than the reference frequency(block S20: No), the condition generation unit 26 maintains theretrieval candidate included in the converted character string as aretrieval candidate (block S22). For example, if determining that theconverted character string “yuu-hatsu-sei-ga-n” (induced cancer) doesnot satisfy the expression (1), the condition generation unit 26maintains the retrieval candidate “yuu-hatsu-sei” (induced) as aretrieval candidate.

On the other hand, if determining that the expression (1) is satisfied,specifically, that the ratio of the part which matches the convertedcharacter string and corresponds to the target character string amongthe words divided by the language processing is equal to or less thanthe reference frequency (block S20: Yes), the condition generation unit26 considers the retrieval candidate included in the converted characterstring as an exclusion candidate (block S24). For example, ifdetermining that the converted character string “ga-n-yuu-hatsu-sei”(cancer-induced) satisfies the expression (1), the condition generationunit 26 changes the retrieval candidate “yuu-hatsu-sei” (induced) to anexclusion candidate. In this embodiment, it is assumed that theconverted character string “ga-n-ken-shin” (cancer medical examination)does not satisfy the expression (1), either, and the retrieval candidate“ken-shin” (medical examination) is also changed to an exclusioncandidate.

The condition generation unit 26 determines whether the determinedconverted character string is the last converted character string (blockS26). The condition generation unit 26 repeats block S20 until theprocessing of block S20 has been performed with respect to all convertedcharacter strings (block S26: No).

If determining that the processing of block S20 has been performed withrespect to all converted character strings (block S26: Yes), thecondition generation unit 26 generates the retrieval condition 34 (blockS28). Here, the condition generation unit 26 determines “yuu-hatsu-sei”(induced) as a retrieval candidate and “yuu-hatsu-sei” (induced) and“ken-shin” (medical examination) as exclusion candidates. Therefore, thecondition generation unit 26 generates the following expression (2) asthe retrieval condition 34, where the symbols in the expression (2) arebased on regular expressions:Retrieval condition: (ga-n)^(yuu-hatsu-sei|ken-shin)  (2), wherein:

^ means that the preceding character does not match the characters inparentheses following this symbol; and

| means “or.” In the above example, “yuu-hatsu-sei|ken-shin” means“yuu-hatsu-sei” or “ken-shin.”

Therefore, the retrieval condition 34 of the expression (2) indicatesthat the character strings “ga-n-yuu-hatsu-sei” (cancer-induced) and“ga-n-ken-shin” (cancer medical examination) are excluded, among thecharacter strings including “ga-n.” Thereby, the condition generationunit 26 generates the retrieval condition 34 of matching the targetcharacter string and not matching the converted character stringsatisfying the condition of the expression (1). The condition generationunit 26 stores the generated retrieval condition 34 into the storageunit 18.

The condition generation unit 26 determines whether a user selects acandidate (block S30). For example, the condition generation unit 26causes the display unit 14 to display the display image 40 illustratedin FIG. 4. In the display image 40, “ga-n” (cancer) in the center is atarget character string. The character or character string displayed onthe left side of “ga-n” (cancer) is a retrieval candidate or anexclusion candidate previous to “ga-n” (cancer). The character orcharacter string displayed on the right side of “ga-n” (cancer) is aretrieval candidate or an exclusion candidate subsequent to “ga-n”(cancer). The check mark in a square on the left side of each characteror character string indicates that the character or character stringconcerned is an exclusion candidate selected by the condition generationunit 26. The character or character string with no check mark in thesquare on the left side of the character or character string is aretrieval candidate set by the condition generation unit 26.

The user selects exclusion candidates by placing or removing a checkmark for each character or character string via the input unit 16 whileviewing the display image 40 concerned. Based on acquiring the selectionof the exclusion candidates from the user (block S30: Yes), thecondition generation unit 26 changes the retrieval condition 34 andstores the new retrieval condition 34 into the storage unit 18 (blockS32). On the other hand, in the case of not acquiring any selection ofthe exclusion candidates (block S30: No), the condition generation unit26 omits the execution of block S32. Thereby, the retrieval conditiongeneration processing ends.

In the retrieval processing, the retrieval unit 28 retrieves a text onthe basis of the retrieval condition 34 stored in the storage unit 18(block S34). The retrieval unit 28 may acquire the retrieval condition34 from the condition generation unit 26. For example, in the case ofperforming retrieval on the text corpus 32 illustrated in FIG. 5, theretrieval unit 28 extracts texts TX1 to TX6 including the targetcharacter string “ga-n” (cancer). Subsequently, the retrieval unit 28excludes the texts TX5 and TX6 including “ga-n-yuu-hatsu-sei”(cancer-induced) and “ga-n-ken-shin” (cancer medical examination) to beexclusion targets, respectively. Thereby, the retrieval unit 28eventually extracts the texts TX1 to TX4. Herewith, the retrievalprocessing ends. The retrieval processing does not need to be performedcontinuously with the retrieval condition generation processing, but maybe performed separately.

As described in the above, in the information processing device 10, thecondition generation unit 26 determines a retrieval candidate as anexclusion candidate in the case where the retrieval candidate isappended to a converted character string in which the percentage of thenumber of target character strings matching the converted characterstring and retrieved based on the language processing to the number oftarget character strings retrieved based on the language processing isequal to or less than the reference frequency. Thereby, the informationprocessing device 10 is able to extract character strings which shouldbe extracted among the character strings including the target characterstring with high accuracy.

For example, “ga-n-yuu-hatsu-sei” (cancer-induced) described in theabove example of the embodiment is not a cancer represented by “ga-n,”but another disease. Therefore, when the target character string is“ga-n” (cancer), the character string “ga-n-yuu-hatsu-sei”(cancer-induced) is not a character string which should be extracted. Inthis case, the condition generation unit 26 generates a retrievalcondition 34 for excluding the converted character strings“ga-n-yuu-hatsu-sei” (cancer-induced) and “ga-n-ken-shin” (cancermedical examination) among the character strings including the targetcharacter string “ga-n” (cancer). Therefore, it is understood that theretrieval condition 34 can be used to exclude character strings whichshould not be extracted. In this manner, the information processingdevice 10 is able to generate a retrieval condition 34 for excludingcharacter strings which should not be extracted among the characterstrings including the target character string so as to improve theextraction accuracy of the character strings.

Moreover, since the condition generation unit 26 generates the retrievalcondition 34 of matching the target character string and not matchingany one of the converted character strings, the information processingdevice 10 is able to extract a character string which is not extractedin the case where the first texts divided into words by the languageprocessing are determined to be retrieval targets.

For example, in the case where “haku-nai-syou-syu-jutu” (cataractsurgery) is retrieved as a target character string by languageprocessing and where “migi-haku-nai-syou” (right cataract) and“syu-jutu” (surgery) are registered as words in the dictionary data 30,a character string “migi-haku-nai-syou-syu-jutu” (right cataractsurgery) in a text is divided into words, “migi-haku-nai-syou” (rightcataract) and “syu-jutu” (surgery), by which the character string“haku-nai-syou-syu-jutu” (cataract surgery) has not been extracted. Onthe other hand, the information processing device 10 extracts allcharacter strings each including the target character string“haku-nai-syou-syu-jutu” (cataract surgery) by pattern matching on thebasis of the retrieval condition 34. Therefore, the informationprocessing device 10 is also able to extract the character string“migi-haku-nai-syou-syu-jutu” (right cataract surgery) as long as itdoes not correspond to an exclusion candidate. Moreover, in the casewhere the target character string is “ma-hi” (paralysis), a characterstring “hidari-bo-shi-ma-hi” (left thumb paralysis) in a text has notbeen extracted since it includes an unknown word “bo.” This is becausethe character string “hidari-so-shi-ma-hi” (left thumb paralysis)includes the unknown word “bo” and therefore is recognized as an unknownword, by which the character string is not divided into words. On theother hand, the information processing device 10 extracts all characterstrings each including the target character string “ma-hi” (paralysis)once on the basis of the retrieval condition 34 and therefore is alsoable to extract “hidari-bo-shi-ma-hi” (left thumb paralysis) as long asit does not correspond to an exclusion candidate.

In the information processing device 10, the condition generation unit26 is able to generate the retrieval condition 34 by using an existingdictionary data 30. Thereby, the information processing device 10 isable to constantly improve the accuracy of the retrieval condition 34 byupdating the dictionary data 30.

In the information processing device 10, the condition generation unit26 causes the display unit 14 to display the display image 40 whichallows exclusion candidates to be selected. This enables the informationprocessing device 10 to visualize the retrieval condition 34 so as toshow the user what retrieval condition 34 is used for the retrieval.

The following describes an example where the aforementioned embodimentis varied.

Determination of Reference Frequency

An embodiment of the determination of the reference frequency in blockS20 will be described. The condition generation unit 26 may make itcondition that the ratio of the part which matches the convertedcharacter string and corresponds to the target character string amongthe words divided by the language processing in the first texts exceedsthe reference frequency. Specifically, the condition generation unit 26may determine whether the following expression (3) is satisfied.Th<#(Ra∩Rx)/#Rx  (3)

The condition generation unit 26 may generate a retrieval condition 34not including a restriction by the converted character string oncondition that the expression (3) is satisfied. In the above embodiment,the converted character string “yuu-hatsu-sei-ga-n” (induced cancer)satisfies the expression (3) with respect to the target character string“ga-n” (cancer), and therefore the condition generation unit 26generates a retrieval condition 34 not including the restriction by theconverted character string “yuu-hatsu-sei-ga-n” (induced cancer), inother words, not excluding the converted character string“yuu-hatsu-sei-ga-n” (induced cancer).

The condition generation unit 26 may determine whether the expression(1) is satisfied on condition that the following expression (4) issatisfied:Th<#(Ra−Rx)/#Rx  (4)

In other words, the condition generation unit 26 may make it conditionthat the ratio of the part which matches the converted character stringand does not correspond to the target character string among the wordsdivided by the language processing in the first texts exceeds thereference frequency. In this case, the condition generation unit 26generates a retrieval condition 34 of matching the target characterstring and not matching the converted character string which satisfiesthe condition of the expression (4).

Generation of Retrieval Condition

An embodiment of the generation of a retrieval condition 34 will bedescribed. FIG. 6 is a diagram illustrating a text corpus including athird text used for the generation of a retrieval condition. Inaddition, the condition generation unit 26 may generate a retrievalcondition 34 of matching the target character string and not matchingthe converted character string on condition that the converted characterstring does not have an attribute by determining whether the convertedcharacter string has the attribute on the basis of the frequency atwhich the converted character string matches at least one third textwith which an attribute depending on the target character string isassociated.

For example, the condition generation unit 26 learns whether the thirdtext has an attribute depending on the target character string bylogistic regression with the frequency at which the converted characterstring matches the third text as an explanatory variable and generates aretrieval condition 34 of matching the target character string and notmatching the converted character string on condition that theexplanatory variable has a negative correlation with that the convertedcharacter string has the attribute.

Specifically, assuming that the target character string is “ga-n”(cancer), the condition generation unit 26 generates the retrievalcondition 34 on the basis of a third text TXm (m=11, --, 21, --, 31, --)of the text corpus 42 illustrated in FIG. 6. Here, an objective variablein a logistic regression analysis is assumed to be a probability ofbeing the target of receiving the payment of insurance proceeds. Inother words, the probability that the converted character string isconsidered as “ga-n” (cancer) is assumed to be an objective variable. Inthe case of being the target of receiving the payment of insuranceproceeds, the objective variable is 1. In the case of not being thetarget of receiving the payment of insurance proceeds, the objectivevariable is 0. Assuming that the objective variable is p and theexplanatory variable of each converted character string is Xn (n=1, 2,--), the relationship between p and Xn is represented by the followingexpression (5):log(p)=α+β1×1+β2×2+β3×3  (5)

For example, “yuu-hatsu-sei-ga-n” (induced cancer) accounts for 80% inall characters in the text TX11, the condition generation unit 26 setsthe explanatory variable X₁ of “yuu-hatsu-sei-ga-n” (induced cancer) inthe text TX11 to 80%. Moreover, if the text TX11 is a text which is thetarget of receiving the payment of insurance proceeds, the conditiongeneration unit 26 sets the objective variable of the text TX11 to 1.Similarly, with respect to other texts TX21 and TX31, the conditiongeneration unit 26 calculates the percentage of “ga-n-yuu-hatsu-sei”(cancer-induced) or “ga-n-ken-shin” (cancer medical examination) in thetext TX21 or TX31 and calculates the explanatory variable X₂ or X₃ of“ga-n-yuu-hatsu-sei” (cancer-induced) or “ga-n-ken-shin” (cancer medicalexamination). Incidentally, the texts TX21 and TX31 are not the targetsof receiving the payment of insurance proceeds and therefore theobjective variable is set to 0. In this manner, the condition generationunit 26 finds a plurality of combinations of an explanatory variable andan objective variable and estimates a coefficient α of each explanatoryvariable illustrated in expression (1) and a coefficient βn associatedwith each converted character string by using the maximum likelihoodmethod which is the estimation method of the known logistic regressionanalysis. If the coefficient βn is positive, the condition generationunit 26 determines that the attribute of the converted character stringis positive. On the other hand, if the coefficient βn is negative, thecondition generation unit 26 determines that the attribute of theconverted character string is negative. The condition generation unit 26generates the retrieval condition 34 of matching the target characterstring and not matching the negative converted character string in theattribute on the basis of these determination results.

Generation of Converted Character String

The following describes an embodiment of the generation of the convertedcharacter string in block S12. The character conversion unit 24 maygenerate a converted character string on the basis of a character stringincluded in a text, instead of words in the dictionary data 30. Forexample, the character conversion unit 24 may generate a convertedcharacter string by retrieving a target character string in the secondtext which is the same as or different from the first text and appendingat least one character located in at least either one of the previousand subsequent positions of the retrieved target character string to thetarget character string. In this case, the character conversion unit 24may generate the converted character string by appending at least onecharacter common to the reference number Stb or more of parts in thesecond text to the target character string. For example, in the casewhere the target character string is “ga-n” (cancer), the characterconversion unit 24 retrieves “ga-n” (cancer) in the second text and addscharacters one by one in at least either one of the previous andsubsequent positions of “ga-n” (cancer). The character conversion unit24 determines the longest character string to be a converted characterstring among the character strings in the case where there are thereference number Stb or more of parts which are the same as thecharacter string with characters added to “ga-n” (cancer).

The character conversion unit 24 may generate a plurality of convertedcharacter strings by appending at least one character for each of theplurality of converted character strings corresponding to a plurality ofinflectional forms with respect to the target character string of aninflectional word. For example, the character conversion unit 24generates converted character strings by appending at least onecharacter of a retrieval candidate to the inflectional forms of thetarget character string.

While the character conversion unit 24 has generated the convertedcharacter strings if the number of character strings each including atarget character string is greater than or equal to the reference numberof words Sta, or greater than or equal to the reference number Stb inthe above embodiment, the generation of converted character strings isnot limited thereto. For example, the character conversion unit 24 mayconsider all character strings each including a target character stringto be converted character strings. Specifically, the characterconversion unit 24 may set the reference number of words Sta and thereference number Stb to 1.

In some embodiments, while the character conversion unit 24 hasextracted retrieval candidates on the basis of the dictionary data 30 orthe text corpus 32 in the above embodiment, the retrieval candidates arenot limited thereto. For example, the character conversion unit 24 mayextract retrieval candidates from character sets each including anarbitrary katakana character. Moreover, the character conversion unit 24may extract retrieval candidates from character string patterns such asChinese numerals.

The character conversion unit 24 may replace at least one character of atarget character string. For example, if the target character string isan English word, the character conversion unit 24 may replace one ormore characters of the target character string. Specifically, thecharacter conversion unit 24 may generate a converted character stringby replacing “virus” with “viral”. Moreover, the character conversionunit 24 may generate a converted character string by performing at leastone of appending and replacement or may generate a converted characterstring by performing both of appending and replacement. In other words,the character conversion unit 24 may perform at least one of appendingat least one character in at least either one of the previous andsubsequent positions of the target character string and replacing atleast one character of the target character string.

FIG. 7 illustrates an example of a hardware configuration of a computer1900 according to this embodiment. The computer 1900 according to thisembodiment is an example of the information processing device 10. Thecomputer 1900 includes a CPU peripheral unit, an input/output unit, anda legacy input/output unit. The CPU peripheral unit includes a CPU 2000,a RAM 2020, and a graphics controller 2075, all of which are mutuallyconnected to one another via a host controller 2082. The CPU peripheralunit also includes a display unit 2080. The input/output unit includes acommunication interface 2030 and a hard disk drive 2040, both of whichare connected to the host controller 2082 via an input/output controller2084. The legacy input/output unit includes a ROM 2010, a memory drive2050, and an input/output chip 2070, all of which are connected to theinput/output controller 2084.

The host controller 2082 mutually connects the RAM 2020 to the CPU 2000and the graphics controller 2075, both of which access the RAM 2020 at ahigh transfer rate. The CPU 2000 operates according to a program storedin the ROM 2010 and the RAM 2020, and controls each of the components.The graphics controller 2075 obtains image data generated by the CPU2000 or the like in a frame buffer provided in the RAM 2020, and causesthe display unit 2080 to display the obtained image data. In place ofthis, the graphics controller 2075 may internally include a frame bufferin which the image data generated by the CPU 2000 or the like is stored.

The input/output controller 2084 connects the host controller 2082 tothe communication interface 2030 and the hard disk drive 2040, both ofwhich are relatively high-speed input/output devices. The communicationinterface 2030 communicates with another device via a network. The harddisk drive 2040 stores, therein, a program such as a display program anddata to be used by the CPU 2000 in the computer 1900.

In addition, the input/output controller 2084 is connected to relativelylow-speed input/output devices such as the ROM 2010, the memory drive2050, and the input/output chip 2070. The ROM 2010 stores a program suchas a boot program executed at a start-up time of the computer 1900and/or a program depending on hardware of the computer 1900 or the like.The memory drive 2050 reads a program or data such as, for example, adisplay program from a memory card 2090, and provides the read programor data to the hard disk drive 2040 via the RAM 2020. The input/outputchip 2070 connects the memory drive 2050 to the input/output controller2084 and also connects various kinds of input/output devices to theinput/output controller 2084 through a parallel port, a serial port, akeyboard port, a mouse port, and the like, for example.

A program to be provided to the hard disk drive 2040 via the RAM 2020 isprovided by a user with the program stored in a recording medium such asthe memory card 2090 or an IC card. The program such as a displayprogram is read from the recording medium, then installed into the harddisk drive 2040 in the computer 1900 via the RAM 2020 and executed bythe CPU 2000.

The program to be installed in the computer 1900 and to cause thecomputer 1900 to function as the information processing device 10includes a language processing module, a character conversion module, acondition generation module, and a retrieval module. Such program ormodules works on the CPU 2000 to cause the computer 1900 to function asthe language processing module, a character conversion module, acondition generation module, and a retrieval module.

Information processing written in these programs are read by thecomputer 1900 and thereby function as a language processing module, acharacter conversion module, a condition generation module, and aretrieval module, all of which are specific means resulting fromcooperation of software and the aforementioned various types of hardwareresources. Moreover, the information processing device 10 specific to anintended purpose is built up by performing computation or processing forinformation in accordance with the intended purpose of the computer 1900in this embodiment by use of such specific means.

In a case where communications are performed between the computer 1900and an external device, for example, the CPU 2000 executes acommunication program loaded on the RAM 2020 and instructs thecommunication interface 2030 on the basis of processing contentsdescribed in the communication program to perform communicationprocessing. Upon receiving the control from the CPU 2000, thecommunication interface 2030 reads out transmission data stored in atransmission buffer region or the like provided in a storage device suchas the RAM 2020, the hard disk drive 2040, the memory card 2090, or thelike and then transmits the data to a network or writes reception datareceived from the network into a receiving buffer region or the likeprovided on the storage device. As described above, the communicationinterface 2030 is allowed to transfer transmission and reception databetween itself and a storage device by a direct memory access (DMA)scheme. Instead of this, the CPU 2000 is also allowed to read data froma storage device of or a communication interface 2030 of a transfersource and then to transfer the transmission and reception data bywriting the data into a communication interface 2030 or a storage deviceof a transfer destination.

In addition, the CPU 2000 causes all of, or a required portion of, datato be read from a file or a database stored in an external storagedevice such as the hard disk drive 2040, the memory drive 2050 (thememory card 2090) or the like into the RAM 2020 by DMA transfer or thelike, and then performs various kinds of processing for the data in theRAM 2020. Then, the CPU 2000 writes the processed data back into theexternal storage device by DMA transfer or the like. In such processing,since the RAM 2020 can be considered as a device in which contents ofthe external storage device are stored temporarily, the RAM 2020 and theexternal storage device or the like are collectively termed as a memory,a storage unit, a storage device, or the like in this embodiment.Various types of information including various types of programs, data,tables, databases and the like in this embodiment is stored in such astorage device and is handled as an information processing target. Itshould be noted that the CPU 2000 is allowed to retain a part of data inthe RAM 2020 in a cache memory and then to read and write the data inthe cache memory. In this case as well, since the cache memory partiallyshares the function of RAM 2020, the cache memory is considered to beincluded in the RAM 2020, a memory and/or a storage device except for acase where the cache memory needs to be distinguished from the RAM 2020,a memory and/or a storage device.

In addition, the CPU 2000 performs, on the data read from the RAM 2020,various types of processing being specified by a sequence ofinstructions of the program and including various types of computations,information processing, conditional judgment, information retrieval andreplacement and the like described in this embodiment, and writes theprocessed data back into the RAM 2020. In a case where the CPU 2000performs conditional judgment, for example, the CPU 2000 determines, bycomparing a variable with the other variable or constant, whether or noteach of various types of variables indicated in the present embodimentsatisfies a condition whether or not the variable is larger, smaller,not less, not greater, equal or the like. In a case where the conditionis satisfied (or the condition is not satisfied), the processing of theCPU 2000 branches to a different instruction sequence or calls asubroutine. In addition, the CPU 2000 may retrieve information stored ina file, a database, or the like in the storage device.

The programs or modules described above may be stored in an externalrecording medium. As the recording medium, any one of the followingmedia may be used: an optical recording medium such as a DVD or a CD; amagneto-optic recording medium such as an MO; a tape medium; and asemiconductor memory such as an IC card, in addition to the memory card2090. Alternatively, the program may be provided to the computer 1900via a network, by using, as a recording medium, a storage device such asa hard disk or a RAM provided in a server system connected to a privatecommunication network or the Internet.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention. Aspects of thepresent invention are described herein with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems), andcomputer program products according to embodiments of the invention. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, can be implemented by computerreadable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

It will be clear to one skilled in the art that many improvements andmodifications can be made to the foregoing exemplary embodiment withoutdeparting from the scope of the present invention.

What is claimed is:
 1. A system for generating a retrieval condition for retrieving a target character string from texts by pattern matching, the system comprising: a memory; and a processor in communication with the memory, the processor being configured to perform operations comprising: dividing a first text into words; generating a converted character string by performing at least one of appending at least one character in at least either one of previous and subsequent positions of the target character string; replacing at least one character of the target character string; generating the retrieval condition for retrieval candidates in the words of the first text, wherein the retrieval condition improves extraction accuracy of the target character string by determining that a retrieval candidate is an exclusion candidate based on the retrieval candidate being appended to the converted character string, and the retrieval candidate matching the target character string and not matching the converted character string based on a ratio of a part of the retrieval candidate which matches the converted character string and corresponds to the target character string is less than or equal to a reference frequency; retrieving the target character string based on the retrieval condition; determining whether a second text has an attribute that depends from the target character string by using logistic regression to identify a frequency at which the converted character string matches the second text as an explanatory variable, wherein the explanatory variable is part of the first text, and wherein the retrieval condition of matching the target character string and not matching the converted character string is generated based on the explanatory variable having a positive correlation with the converted character string that has the attribute; and determining whether a third text has an attribute that depends from the target character string by using logistic regression to identify a frequency at which the converted character string matches the third text as the explanatory variable, wherein the retrieval condition of matching the target character string and not matching the converted character string is generated based on the explanatory variable having a negative correlation with the converted character string that has the attribute. 